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This study examined the impact of instructional variables on team learning outcomes and collaboration 
between online- and blended groups. Differences in team learning collaboration and instructional factors 
were found between the groups and significant relationships existed between them. From the findings, 
implications and discussions to improve existing practices of online or blended instruction were made. 

Keywords: Team Learning Performance/Collaboration, Online Learning, Blended Learning 

Online instruction and blended instruction are two widely used instructional delivery methods in academic and 
corporate settings. Online instruction is viewed as the most promising technology-based solution for providing up- 
to-date knowledge to distributed individuals while onsite instruction is recognized as superior in networking and 
learning complex contents (Yoon & Lim, 2007). With online technologies capable of coordinating remote resources 
and individuals in a cost-effective manner, changes and mixes of instructional sequences and methods are possible 
for enhancing learners' educational experiences, especially regarding learners’ interactions with contents and peers. 
However, management challenges such as how to facilitate the process of social learning or how collaboration 
affects learning outcomes between different delivery modes are less well-known. How learning should take the form 
of collective knowledge and is iterative through the process of exchanging practices are well-documented in the 
literature (Lave & Wenger, 1991; Nonaka & Konno, 1998). Improvements in learning, satisfaction, collaboration, 
application to work, and administrative efficiency are all very important indicators to be measured. Equally 
important are variables related to designing and implementing teams whose members share the same goal and 
collaborate to enhance learning, group outcomes, and work processes. As both forms of groups are growing in 
numbers and interests, comparative empirical studies that are based on sound theoretical and methodological 
frameworks are in greater demand. 

Theoretical Framework 

Online Instruction 

Online instruction is a form of virtual learning and instructional environment which facilitates participants’ 
cognitive, constructive, and communicative learning needs. It commonly takes place in computer-networked 
environments (Kearsley, 1997) and can deliver what educators recognize as essential components of learning 
environments to offer proper contexts, tools, resources, and learning-facilitating scaffolds which initiate and guide 
authentic problem-solving processes (Hannafin, Land, & Oliver, 1999). The term also is used to reflect the historical 
expansion of major distance learning technologies which started from correspondence and evolved through 
transmittal broadcasting systems, multimedia-rich Internet, and web-based audio and video conferencing systems 
(Morabito, Sack, & Bhate, 1999). Benefits of online education include expanding educational opportunities, 
promoting self-directed, active, and reflective individuals (Berge, 2002), saving cost for training, and shifting the 
focus of training from learning to performance (Rosenberg, 2006). The fifth annual report on the state of U.S. online 
learning, supported by the Albert P. Sloan foundation, indicates that almost three and a half million students were 
taking an online course during the fall 2006 semester (Allen & Seaman, 2007). However, online instruction has been 
challenged by limitations in the capability to engage learners in learning events and deep-level learning and high 
expectation of self-disciplined and motivated learners. 

Blended Instruction 

Blended instruction is most commonly defined as the mix of traditional onsite instruction with any innovative 
learning technologies (Thorne, 2003). Variations of definitions exist that are noteworthy. Rossette (2003) claims 
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blending as a mix of instructional methods with delivery media, which includes onsite instruction. A lot of 
academics support the definition of blending as the substantial integration of technologies into onsite instruction 
Picciano (2006). Recently, Yoon and Lim (2007) maintained that blending is most easy to understand by 
recognizing the widespread use and strengths of onsite instruction and simultaneously leveraging simplified 
categories of major technologies (i.e., Web/Computer-Based Training, Web/Knowledge Management 
(KM)/Electronic Performance Support System (EPSS), Web/Asynchronous, and Web/Synchronous) to support core 
business performance and learning targets. This view is similar to Rosenberg’s (2006) proposal of conceptualizing 
blending as leveraging technological and social networks (e.g., EPSS, KM, communities of practices, and learning 
solutions) aimed at aligning organizational and individual learning. Researchers strongly suggest to adopt this 
approach claming that blending improves learner satisfaction and educational experiences from easier access to 
knowledge, increased social presence, ease of updating contents, and technology-enhanced learning experiences 
(Osguthorpe & Graham, 2003; Reece & Lockee, 2005). 

As the term blended implies, different patterns exist to mix various delivery media and instructional methods or 
learning theories. Singh and Reed (2001) suggested that six combinations of blended types of instruction are 
possible: (a) offline and online learning, (b) self-paced, live, and collaborative learning, (c) structured and 
unstructured learning, (d) custom content with off-the-shelf content, (e) work and learning, and (f) ingredients of the 
blend synchronous physical formats, synchronous online formats, and self-paced, asynchronous formats. Carmen 
(2002) noted that in technology-utilizing blended environments, instructors should be competent in aligning live 
events, self-paced learning, collaboration, assessment, and learning support materials. We prefer to use Driscoll's 
(2002) simpler technology classification of online instruction consisting of Web/CBT, Web/KM-EPSS, Web/Virtual 
Asynchronous Classrooms, and Web/Virtual Synchronous Classrooms and support the importance of instructors’ 
competence in course designing and interaction facilitating skills (Aragon & Johnson, 2002). 

Instructional Factors 

Reviewing the literature of online and blended instruction has led to the identification of major instructional 
variables to consider. These include (a) the quality of the instructor represented as instructor presence (Anderson, 
Rourke, Garrison, & Archer, 2001), competencies (Aragon & Johnson, 2002), facilitation skills (Palloff & Pratt, 
1999) and design (Clark & Taylor, 1992; Hirumi, 2002; Lee, Owens, & Benson, 2002), (b) the quality and variety of 
contents, problems, and activities for relevance and interests (Garavaglia, 1993), (c) learner characteristics, such as 
technology skills, readiness, self-directedness (Yang & Cornelious, 2005), action planning (Foxon, 1997) and 
commitment (Lim, Yoon, & Morris, 2006; Oh & Lim, 2005), and (d) learner support systems, such as tutors, 
counselors, registration, financial aid, and libraries (Huett, Moller, & Young, 2004; LaPadula, 2001; Muilenburg & 
Berge, 2002). This picture shows that numerous conditions and strategies are available and can be manipulated by 
instructors for directly or indirectly affecting course outcomes. It also shows the challenge of selecting and working 
with informed practices that critically matter to the particular contexts and targeted learners. 

Evaluating Team Learning Performance 

Collaborative learning groups that share the same goal and work protocols have been recognized as important 
for leveraging collective knowledge and distributed resources (Johnson & Johnson, 2006). In evaluating learning 
outcomes at a group or team level (we use these terms interchangeably, although distinctions are often made on 
teams as imposing greater trust, accountability, and shared work protocols), a scheme that is commonly used for 
evaluating individual learning should be applied with caution. Low-level assessments on individuals measure the 
level of satisfaction and mastery of contents, while higher-level forms attempt to capture the level of behavioral 
changes, work application and business impacts (Sullivan, 2002). Similarly, team or group level learning can be 
assessed at the low level through the acquisition of knowledge and member satisfaction, or at higher levels, such as 
enhanced work processes, but for these measures to be applied at the team level, proper sampling methods, such as 
randomization between different types of groups and greater numbers of groups (at least larger than 10 in each 
group) are required, if quantitative analyses are to be done. Additionally, a learning group requires members to 
communicate to introduce potential outcome or intermediate variables, such as the level of trust, work norms, and 
collaboration. 

Purpose of the Study 

In view of the widespread use and growing interests of online and blended learning groups, this study purported to 
examine whether these two groups differed in learning outcomes and how important instructional variables 
established in the literature impacted them. The following research questions were developed: 
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Is there a difference in team learning outcomes and team’s perception of instructional conditions between 
online and blended learning groups? 

2. How instructional variables are related to learning and performance measures? 

3. To what extent do instructional variables predict team learning and performance? 

Methodology 

Participants and Setting 

Participants in this study were 222 undergraduate students (73 male and 149 female) enrolled into a course in 
program evaluation offered by an HRD program at a southeastern university. Among them, 156 were between 18-19 
years old, 43 were between 20-29 years old, and 23 were 30 years old or above. Regarding job status, 54 were 
fulltime workers, 98 were part time workers, and 70 were fulltime students. The course has an approximate 
enrollment of 20 students in each semester. The data were collected over 17 semesters from the same course 
between 2000 and 2005. The course was developed to teach curriculum contents on learner and program evaluation 
for HRD undergraduate students. The course was delivered completely online for the first three and half years, and 
then, modified for the following two years to take the form of a blended course. This history of the course resulted in 
44 online learning groups and 25 blended learning groups. 

Regarding the online delivery method, the instructor developed thirteen online learning modules and the 
workload of one module was equivalent to that of one week’s classroom instruction. Learning modules provided 
subject content in learner and program evaluation. Several interactive learning activities including online 
discussions, case study analyses, and online tests and surveys were utilized within the modules to provide the 
learners with opportunities to apply learned content during learning. All learners were asked to attend the first and 
last class meeting for course orientation and group project presentation respectively. 

The modified blended format offered one (occasionally two) out of three onsite classroom instruction every 
week to accommodate the university academic system. This change originated from the mutual interests of the 
university (having the majority of part-time and fulltime students in the course present on campus) and the instructor 
in enhancing pedagogy. After each week's classroom instruction, the learners were asked to complete online 
learning modules which included various learning activities, such as a review of the content, links to learning 
resources, group discussions and project. Care was given to create an equal amount of study hours between these 
two different delivery formats. 

Instrument and Procedure 

After confirming the viability of online questionnaire (MacElroy, 1999; Raziano, Jayadevappa, & Valenzula, 
2001), team learning and performance outcome measures and instructional variables based on the literature review 
were collected. Participants were required to complete questionnaires. All of the question items except feedback- 
soliciting open-ended items were required to successfully complete each questionnaire. Questionnaires for learning 
included pre- and post measures on course contents (labeled as actual learning and viewed as a knowledge test) and 
competencies for eighteen course objectives (labeled as perceived learning). The competency measure utilized a five 
point Likert-type scale learning (1-do not understand to 5-completely understand). One questionnaire that measured 
their level of interests in application to work was implemented at the end of the course only. For fulltime students 
who did not possess directly applicable work experiences, the nature of the course crossed over to others, such as 
instructional design and learning theories, thus directions asked to imagine their application within his/her chosen 
academic discipline. This also used a five point Likert-type scale with 1 indicating none and 5 showing a very 
frequent use. 

Participants also responded to 17 close-ended questions that measured their perception about the course in the 
areas of the quality of the instructor, learning activities, the level of learning support, and perceived workload. All 
question items in this category also used a five point Likert-type scale. For instructor quality, five items asking 
instructor’s subject matter expertise, responsiveness to students’ questions, questioning techniques, use of examples, 
and an ability to make clear presentations were included. For learning activities, five question items asking students’ 
perception of the quality of course discussion, case studies, assignments, group and individual projects were 
included. For learning support, three question asked students’ perceived feedback of assignments, study questions, 
and technical support. For workload, four items assessed perceived weekly hours spent on the course, assignments, 
and how those hours compared to other online and onsite courses of equal credit hours. Lastly, to measure the level 
of team performance, several items assessed the members’ contribution toward learning as a group and completing 
group assignments and projects (1-not contributed to 4-fully contributed). This was collected at the end. Examining 
the result of Shapiro-Wilk's normality test with alpha level at .5 and skewness and kurtosis indicated that normality 
assumptions were met for all items. 
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Data Analysis 

Basic descriptive statistics analyzed the test scores and the perceived degree of learning, application of learning, 
and selected instructional variables of the course. The independent-sample /-test was used to compare how those two 
groups, online and blended learning groups differed in learning outcomes and perceived instructional conditions. 
Correlation analyses were performed to test inter-relationships among instructional variables and learning outcome 
measures. Finally, the stepwise regression analysis was performed to explore the influence of instructional factors on 
various learning outcome measures. These exploratory approaches were taken over MANOVA and covariate 
analyses due to the lack of clarity in conceptual relationships between learning and collaboration measures and the 
lack of randomization and experimental control to justify identifying treatment effects. The result of correlation 
analysis and the Exploratory Factor Analysis led us to confirm one dimensionality of chosen instructional variables 
to comprise the overall instructional quality. 

Findings 

Before we examined the differences between online and blended learning groups via group-averaged scores, 
descriptive analyses examined the mean and standard deviation of different groupings, such as by gender, age, and 
two delivery modes by individuals. The overall results did not show any grouping to be a potential moderator, thus 
we proceeded to perform the independent-sample /-test. The /-test indicated that online and blended learning groups 
were not different in terms of any learning outcome measures (Table 1). The largest difference in outcome measures 
was detected in the level of perceived contribution to the group work. Also, blended learning groups reported a 
significantly higher score on member contributions (M - 3.81, SD = .26) than members of the online delivery groups 
( M = 3.64, SD = .35). Interestingly, two groups displayed larger differences in the perceptions of instructional 
variables in the course rather than differences in learning outcomes. Overall, blended learning groups reported more 
positive perceptions regarding the quality of the instructor, learning activities, and learning supports. They also 
reported comparatively higher scores in the perceived course difficulty, study time, and course workload. Among 
them, difficulty level, workload, study time, and learning support indicated statistically significant differences. 



Table 1, Comparisons on Instructional Variables and Learning Outcomes between Online and Blended Groups 





Online Delivery Group 
(n = 44) 


Blended Delivery Group 
(n = 25) 


p- value 


Instructional factors 
- Instructor quality 


3.73 (.98) 


3.83 (.86) 


.708 


- Learning activity 


3.53 (.88) 


3.85 (.79) 


.208 


- Difficulty level 


3.29 (.94) 


3.33 (.87) 


.009 


- Work load 


3.65 (.86) 


3.81 (.81) 


<.001 


- Study time 


3.57(1.00) 


3.73 (.87) 


<.001 


- Learning support 


3.48 (.86) 


3.65 (1.06) 


<.001 



Learning outcomes 
- Perceived learning ( PL) 


3.89 (.40) 


3.88 (.30) 


.910 


- PL gain 


.82 (.53) 


.78 (.50) 


.742 


- Posttest 


11.23 (2.79) 


11.52 (2.21) 


.607 


- Actual learning gain 


3.12 (2.38) 


3.00 (2.31) 


.861 


- Learning application 


3.76 (.39) 


3.72 (.35) 


.665 


Team collaboration 


3.64 (.35) 


3.81 (.26) 


.027 



We also compared learning outcome variables between those who had no complaints with team collaboration 
(defined as scoring 4 or above out of five) and those with some complaints (defined as scoring 3 or below out of 
five) regardless of the course delivery format. The learner groups with no complaints reported significantly higher 
mean scores in perceived learning (n = 129, M= 3.95, SD = .51, p < .05), posttest (n = 85, M= 12.63, SD = 3.35, p < 
.01), and learning application ( n = 129, M= 3.80, SD = .61, p = .05) than those with some complaints (n = 86, M = 
3.78, SD = .67 on perceived learning, n = 43, M = 10.79, SD = 3.35 on posttest, and n = 86, M= 3.63, SD = .71 on 
learning application). 
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Relationships between Instructional Variables and Learning Outcome Measures 

Results from the correlations between instructional and outcome variables showed a wide range of relationships 
(Table 2). Overall, a strong positive relationship was found between the raw and the gain scores within each 
measure of comprehension (r = .859, p < .01) and competence (r = .545, p < .01). This helped eliminate the concern 
of the regression toward the mean phenomenon to make each score eligible for a valid dependent measure with our 
preference placed on the use of knowledge tests over perception data. This is further supported from the absence of a 
significant relationship between those two measures whether the raw or the gain score are used. It was not surprising 
to find strong correlations between perceived course difficulty, workload, and study time (p < .01). The relationship 
between learning support with various outcome measures was very weak with none of them showing any statistical 
significance. Neither the result of the posttest score or gain score on it from the pretest were statistically significant 
in regards to any instructional variables, although a low to moderate positive relationships were detected in 
comparison with perceived learning (r = .185 ~ .291). Learners’ willingness to apply course learning to work were 
highly correlated with their perception of the quality of the instructor, activities, and perceived learning, but its 
relationship with the knowledge test was almost non-existent. Lastly, perception of team collaboration was 
positively correlated with learning support (r = .339, p <.01 ) and perceived learning (r = .296, p < .05), negatively 
correlated with course difficulty (r = -.318,/? < .01), workload (r = -.212), and study time (r = -.266, p < .05). 



Table 2. Correlations between Instructional Variables and Learning Outcome Measures 





Perceived 

learning 


Perceived 
learning gain 


Posttest 


Actual 
learning gain 


Application 


Collaboration 


Instructor quality 


.407** 


.403** 


.180 


.214 


.484** 


.059 


Activity 


.504** 


.340** 


.115 


.184 


.578** 


.124 


Course difficulty 


.138 


.247* 


.073 


.217 


.258* 


-.318** 


Workload 


-.095 


.047 


-.116 


.001 


-.057 


-.212 


Study time 


.214 


.242* 


.056 


.173 


.263* 


-.266* 


Learning support 


.167 


.001 


.080 


.149 


.190 


.339** 


Collaboration 


.296* 


.181 


-.145 


-.174 


.173 





*Significant at the 0.05 level (2-tailed). **Significant at the 0.01 level (2-tailed). 



Influence of Instructional Factors on Team Learning Outcomes and Collaboration 

The exploratory nature of this study placed team collaboration as a separate dependent measure and explored 
multiple learning outcome measures, such as raw and gain-scores in knowledge tests, perceived competence, and a 
post-measure of learning application. With the findings of no significant differences in all outcome measures 
between online and blended learning groups, stepwise regression analysis was performed. Results showed that either 
the quality of the instructor or the activity was selected as the primary variable (due to a very high correlation 
between them, when one was selected, the other was eliminated) and study time additionally contributed to account 
for the variation of adjusted perceived learning (SR 2 = .366, p < .01) or its gain score (SR 2 = .243, p < .01). 
Examining the last column indicates that changes in the amount of study time may result in greater changes in 
perceived learning, but caution is required since all instructional variables were not captured as continuous data. 
These variables were also very useful in predicting almost half of the variation in their willingness to apply to work, 
but none of them made an entry into predict the outcome of team collaboration. Students’ study workload seemed to 
negatively influence team level learning outcomes in the perceived learning, perceived learning gain, and learning 
application while the difficulty level of the course tended to negatively influenced team’s collaboration during 
learning. 
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Table 3. Stepwise Regression of Learning Outcomes for All Groups 



Learning Outcomes 


R 2 


SR 2 


Predictors 


P 








Learning activity 


.265 


Perceived learning 


.398 


.366 


Study time 


.316 








Workload 


-.244 








Instructor quality 


.270 


Perceived learning gain 


.281 


.243 


Study time 


.453 








Workload 


-.354 








Learning activity 


.314 


Learning application 


.500 


.474 


Study time 


.348 








Workload 


-.256 


Team collaboration 


.183 


.155 


Learning support 


.123 








Difficulty level 


-.138 



Discussion and Implications for HRD Education 

While various forms of learning groups are frequently practiced, empirical research efforts are rather scarce 
( Arbaugh, 2005). Findings from this study indicated that learning outcomes were not significantly different between 
online and blended learning groups. However, these two groups showed a differing level of perceptions toward 
some of the important instructional variables, such as course workload, study time, and learning support. Although 
not statistically significant, these two groups also showed differing degree of perceived course difficulty. This study 
also found instructional variables, such as the quality of the instructor or course activities and study time to be very 
useful in predicting perceived learning outcomes, but learning support only was useful in predicting the success of 
team collaboration. Although online and blended learning groups did not differ in any measured learning outcomes 
in terms of statistical significance, it was also interesting to find a slightly higher score on team collaboration and 
instructional variables from the blended learning groups (Table 1). Such findings as team level learning outcomes 
are influenced by instructional variables in comparison to delivery forms (e.g., online and blended groups) that 
support the greater importance of instructional methods over media, such as different group forms in this study 
(Clark, 1990). 

Findings from this study point to a couple of major themes of discussion that is important for improving current 
practices of utilizing learning groups through online and blended formats. First, selecting qualified instructors and 
designing or working with learning activities that are meaningful and applicable seem to be most important. 
Recommended strategies include: (a) making learning content applicable (Baldwin & Ford, 1988); (b) utilizing 
reflective learning activities to apply learning content to personal situations (Clark & Taylor, 1992); and (c) 
facilitating a structured learning process by step-by-step guided practices and individuals’ independent practice to 
reinforce learning (Lim, 2002). 

Second is the importance of learning support for team collaboration in technology-based groups. Online 
instruction has been challenged for the limited capability to engage learners in learning events. Variables such as the 
quality of the instructor and the quality of the learning activities were not helpful in understanding the level of team 
collaboration, but the result of the correlation and regression analyses showed that perceived workload, difficulties, 
and study time negatively correlated to the construct. And learning support was the only successful predictor in team 
collaboration variation. Findings supported that team collaboration was higher in blended learning groups at p. <.05 
level. Daniels and Moore (2000) suggested blended learning as a potential solution for improving learners’ 
perceived isolation in online environments through member collaborations. Researchers showed that a lack of 
belonging sense to learning communities keeps learners from developing shared feelings and emotional comfort in 
online environments (Oh & Lim, 2005, Rovai, 2002). To address this in online environments, Fortaine (2002) 
suggested that delivering vivid learning experiences and enhancing social presence as crucial. Similarly, Lim (2002) 
suggested several instructional techniques for online instruction including: (a) providing immediate feedback on 
learners’ questions and timely technical support; (b) asking short questions checking the understanding of major 
learning content at frequent intervals during instruction; (c) sending learners' learning progress report on a regular 
basis to promote learners’ motivation for learning achievement; and (d) using humor so the learners feel emotionally 
refreshed and engaged. Onsite instruction can have a place to manage such practices more efficient and immediate. 
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Muilenburg and Berge (2002) suggested more comprehensive institution-wide support, such as supportive 
administration, organizational change management, technical support, promotion of social interactions and open 
access, program evaluation, and student support services. 

Limitations and Future Research 

Using multiple dependent measures in this study (e.g., perceived competence, knowledge tests, and volition of 
application) showed that perception data inflated the results of both the correlations and the regression analysis. 
Before any generalizations are made upon the potential effectiveness of instructional variables in predicting group- 
level learning outcomes, refinements in measurement through using more objective measures, such as knowledge 
tests, improvement in work processes, and quantifiable accomplishments will be helpful. The data from a single 
population in a university setting may also reflect the unique institutional characteristics to the study findings. 

Enhancing team level learning performance and learning collaboration is an important research area within the 
HRD discipline. In future studies, various refinements to the current design are warranted. Such refinements will be 
feasible through approaches, such as careful inclusion of important grouping categories (e.g., group work 
experiences, preferences and attitudes toward group-based learning), randomization of members across different 
group formats, if settings permit, and replication efforts across different topics, settings, discipline, and learners. 
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